Cs-621 Theory Gems

So far, we have seen streaming algorithms for two important variants of Lp-norm estimation problem: L0-norm estimation (the distinct elements problem) and L2-norm estimation. We also noted that the L1norm estimation problem (at least, when we do not allow element deletions) corresponds to just computing the length of the stream and thus can be trivially solved in O(log n) space. Therefore, the next natural step would be to try to approach the task of L∞-norm estimation, i.e., estimating the frequency of the most frequently occurring element. Unfortunately, as mentioned last time, one can show (and we will see it soon) that L∞-norm estimation requires Ω(m) space, which is completely prohibitive from the-point of view of streaming algorithms. This is rather disappointing, as ability to efficiently compute this very fundamental statistic of the data streams would be very valuable. Fortunately, despite this negative result, there is still hope for getting something useful here. Namely, as we will see later, the L∞-norm estimation problem instances that are used in the Ω(m) lowerbound are all corresponding to a situation in which every element (including the most frequent one) has only very few (in fact, at most two) occurrences in the stream. However, this kind of instances are not really that interesting from the-point of view of the intended applications of an L∞-norm estimation algorithm. The scenarios that we would really like to address are the ones in which the most frequent element appears with substantial frequency (think, e.g., about a router trying to detect a Denial-of-Service attack). As it turns out, once we reformulate our problem to allow it to not provide any meaningful answer when there is no very frequently appearing elements, the Ω(m) lowerbound does not hold anymore and one can obtain very satisfactory algorithms for such scenarios.